Image based rendering allows the synthesis of an image for a virtual point of view from a collection of camera images. The point of view of a camera may also be referred to as a ‘viewpoint’. For example, where a subject is surrounded by a ring of cameras, it is known that a new (virtual camera) view of the subject, corresponding to a position in between (real camera) captured views, can be synthesised from captured views if there is sufficient knowledge of camera configuration and a scene.
In recent times, the ability to synthesise an arbitrary viewpoint has been promoted for the purpose of “free viewpoint” video where a viewer is able to actively adjust camera viewpoint to his or her preference within constraints of a capture system. Alternatively, a video producer may employ the free viewpoint technology to construct an “optimal” viewpoint for a passive broadcast audience. In the latter case, considerable flexibility is afforded to the producer such that timely creation of optimal viewpoints, especially for live events such as sports, may be difficult simply because of the range of possibilities available.
In several examples of a viewpoint determining system, subjects within an arena are identified and tracked, possibly using wearable tracking devices that provide real-time information about the location of a subject. The provided information is used, in combination with other attributes of the subject (e.g., including complex knowledge like the relative location of limbs, the centroid of the body etc.) to create a virtual camera that is tethered to and frames the subject. While such a viewpoint determining system is able to create a tracking camera, the viewpoint determining system fails to produce broadcast quality footage in cases where there are interactions between subjects or other events that comprise more than a single subject.
A need exists for methods and systems capable of generating appropriate synthetic viewpoints automatically.